On my list of things I expected to be doing in 2026, this was certainly not at the top, but here we are. Nomadnet and Reticulum are more than a hobby for me at this point; with the state of the west as it is, the only real method I have for managing anxiety is to work on things that maybe, just maybe can make a difference for somebody.

Before I get into the details, here's the link to my version of a Nomadnet Search Engine: Sunspotter. The branding is probably going to change at some point, but I have more important things to improve, like search accuracy and additional user-friendly features, both for the query-ers and page owners.

- Recursive local link crawling

- Avoiding duplicate pages

- Automatic rate-limiting

- ONNX embedding vector generation

- LanceDB page content, metadata, and embedding storage

- Improved sentence parsing and text cleaning

- Reranking on query (currently just vector similarity)

- Query inference improvements

- File-diff tracking, maintaining a log of pages with recent changes

- "Wayback Machine"-style recording of early NomadNet, for posterity

- Closed-loop feedback for improving accuracy over time

- Illegal content reporting and blacklisting

Well, aside from it being fun?? Hmm. Perhaps I'll point to a couple of the existing ones, from which I took some inspiration:

- The Waystone went offline a couple of months ago for me. I'm not sure if they just left the Testnet, but as effective as it was, it is not longer available.

- The Nomad Index has been live for a while, and the accuracy of the search engine is much better than mine at present. However, the page availability has been hit-or-miss for me, and there are some issues in the native NomadNet client with getting the search function itself to work. That being said, it has worked for me, and the performance is excellent.

So I suppose it's fairly obvious why I started the project. Somebody had to.

There are a number of risks incurred when attempting to build a piece of software that could become a significant piece of infrastructure. I'm not the best software developer in the world, so finding all of the things that can go wrong has taken some time. Here is an incomplete list (please comment if you think of something I may have missed!):

Possibly the most-important characteristic of the crawling service is to avoid bogging down the network in any way. This is surprisingly not that difficult to do, as there are only hundreds of pages on nomadnet at present (Jan 2026).

Still, future-planning is necessary. My current implementation handles this in a relatively naive way; a delay of 10x the page request time rolling average (N=5) is included between all page requests. Intuitively, if the path I was using to query the page was the only interface that node had to the wider net, I would avoid loading that node more than 10% of its total bandwidth. This should allow it to simultaneously serve other users even if the site is currently being indexed.

Another critical piece of the puzzle is the re-indexing period for pages and resources. My current setup just hard-codes a 1 week delay between crawls, but there are ways to make this smarter; I might just not be smart enough to figure them out myself---please let me know if you have an idea for an improvement!

This might not really be an issue for now, but it's something I want to plan ahead for. The risk here is twofold:

- Users don't want to receive suggestions for pages which contain illegal or disturbing content.

- I don't want to store and propagate illegal content, whether it's threats of terrorism or CSAM. This is mostly from a personal morality standpoint.

I'm hoping to have some level of automatic filtering for certain types of content, but if a node owner does *not* want a certain page (or even their entire domain) to be indexed, they should have the ability to tell my crawler to avoid those pages.

This might look like the following:

#!c=300,#!i=False

The cache directive should still be respected by the browsers, and the 'i' directive (index) would be respected by my crawler (and other indexing engines that decide to add this feature). This feature isn't *yet* added to my code.

I use a local model (a quant of Qwen3-Next 80B A3B) to assist with Cline inside of Code-OSS (open-source VScode). The mix of hand-written code to generated code is around 50-50, but the initial version of a lot of the tools is purely generated. My reason is simple; I don't really know how a lot of the necessary pieces of a search engine work---or I didn't before starting this project.

I had some experience with generating embeddings and performing reranked queries from my Wiki Bot project, which will be getting its own blog post soon! However, databases were (and are still) a bit of a black box to me. One of my learnings from the Wiki Bot project was that ChromaDB is not to my taste. It has to store the entire database in-memory, which was infeasible with the entirety of English Wikipedia. Additionally, it has some icky opt-out telemetry enabled by default. My firewall caught this, but it's not a good feature.

There are still several serious performance issues which will affect the accuracy of returned results. Additionally, the database is not running on the final machine (yet), as I need to rebuild the ONNX library from source to get it working.

thanks meowing~ learned a new word today <3 zased